Comparing the performance of FA, DFA and DMA using different synthetic long-range correlated time series

Notwithstanding the significant efforts to develop estimators of long-range correlations (LRC) and to compare their performance, no clear consensus exists on what is the best method and under which conditions. In addition, synthetic tests suggest that the performance of LRC estimators varies when using different generators of LRC time series. Here, we compare the performances of four estimators [Fluctuation Analysis (FA), Detrended Fluctuation Analysis (DFA), Backward Detrending Moving Average (BDMA), and centred Detrending Moving Average (CDMA)]. We use three different generators [Fractional Gaussian Noises, and two ways of generating Fractional Brownian Motions]. We find that CDMA has the best performance and DFA is only slightly worse in some situations, while FA performs the worst. In addition, CDMA and DFA are less sensitive to the scaling range than FA. Hence, CDMA and DFA remain "The Methods of Choice" in determining the Hurst index of time series.Comment: 6 pages (including 3 figures) + 3 supplementary figure

Estimation of traffic matrices for LRD traffic

The estimation of traffic matrices in a communications network on the basis of a set of traffic measurements on the network links is a well known problem, for which a number of solutions have been proposed when the traffic does not show dependence over time, as in the case of the Poisson process. However, extensive measurements campaigns conducted on IP networks have shown that the traffic exhibits long range dependence. Here two methods are proposed for the estimation of traffic matrices in the case of long range dependence, their asymptotic properties are studied, and their relative merits are compared

Revisiting detrended fluctuation analysis

Half a century ago Hurst introduced Rescaled Range (R/S) Analysis to study fluctuations in time series. Thousands of works have investigated or applied the original methodology and similar techniques, with Detrended Fluctuation Analysis becoming preferred due to its purported ability to mitigate nonstationaries. We show Detrended Fluctuation Analysis introduces artifacts for nonlinear trends, in contrast to common expectation, and demonstrate that the empirically observed curvature induced is a serious finite-size effect which will always be present. Explicit detrending followed by measurement of the diffusional spread of a signals' associated random walk is preferable, a surprising conclusion given that Detrended Fluctuation Analysis was crafted specifically to replace this approach. The implications are simple yet sweeping: there is no compelling reason to apply Detrended Fluctuation Analysis as it 1) introduces uncontrolled bias; 2) is computationally more expensive than the unbiased estimator; and 3) cannot provide generic or useful protection against nonstationaries

On the nature and impact of self-similarity in real-time systems

In real-time systems with highly variable task execution times simplistic task models are insufficient to accurately model and to analyze the system. Variability can be tackled using distributions rather than a single value, but the proper charac- terization depends on the degree of variability. Self-similarity is one of the deep- est kinds of variability. It characterizes the fact that a workload is not only highly variable, but it is also bursty on many time-scales. This paper identifies in which situations this source of indeterminism can appear in a real-time system: the com- bination of variability in task inter-arrival times and execution times. Although self- similarity is not a claim for all systems with variable execution times, it is not unusual in some applications with real-time requirements, like video processing, networking and gaming. The paper shows how to properly model and to analyze self-similar task sets and how improper modeling can mask deadline misses. The paper derives an analyti- cal expression for the dependence of the deadline miss ratio on the degree of self- similarity and proofs its negative impact on real-time systems performance through system¿s modeling and simulation. This study about the nature and impact of self- similarity on soft real-time systems can help to reduce its effects, to choose the proper scheduling policies, and to avoid its causes at system design time.This work was developed under a grant from the European Union (FRESCOR-FP6/2005/IST/5-03402).Enrique Hernández-Orallo; Vila Carbó, JA. (2012). On the nature and impact of self-similarity in real-time systems. Real-Time Systems. 48(3):294-319. doi:10.1007/s11241-012-9146-0S294319483Abdelzaher TF, Sharma V, Lu C (2004) A utilization bound for aperiodic tasks and priority driven scheduling. IEEE Trans Comput 53(3):334–350Abeni L, Buttazzo G (1999) QoS guarantee using probabilistic deadlines. In: Proc of the Euromicro confererence on real-time systemsAbeni L, Buttazzo G (2004) Resource reservation in dynamic real-time systems. Real-Time Syst 37(2):123–167Anantharam V (1999) Scheduling strategies and long-range dependence. Queueing Syst 33(1–3):73–89Beran J (1994) Statistics for long-memory processes. Chapman and Hall, LondonBeran J, Sherman R, Taqqu M, Willinger W (1995) Long-range dependence in variable-bit-rate video traffic. IEEE Trans Commun 43(2):1566–1579Boxma O, Zwart B (2007) Tails in scheduling. SIGMETRICS Perform Eval Rev 34(4):13–20Brichet F, Roberts J, Simonian A, Veitch D (1996) Heavy traffic analysis of a storage model with long range dependent on/off sources. Queueing Syst 23(1):197–215Crovella M, Bestavros A (1997) Self-similarity in world wide web traffic: evidence and possible causes. IEEE/ACM Trans Netw 5(6):835–846Dìaz J, Garcìa D, Kim K, Lee C, Bello LL, López J, Min LS, Mirabella O (2002) Stochastic analysis of periodic real-time systems. In: Proc of the 23rd IEEE real-time systems symposium, pp 289–300Erramilli A, Narayan O, Willinger W (1996) Experimental queueing analysis with long-range dependent packet traffic. IEEE/ACM Trans Netw 4(2):209–223Erramilli A, Roughan M, Veitch D, Willinger W (2002) Self-similar traffic and network dynamics. Proc IEEE 90(5):800–819Gardner M (1999) Probabilistic analysis and scheduling of critical soft real-time systems. Phd thesis, University of Illinois, Urbana-ChampaignGarrett MW, Willinger W (1994) Analysis, modeling and generation of self-similar vbr video traffic. In: ACM SIGCOMMHarchol-Balter M (2002) Task assignment with unknown duration. J ACM 49(2):260–288Harchol-Balter M (2007) Foreword: Special issue on new perspective in scheduling. SIGMETRICS Perform Eval Rev 34(4):2–3Harchol-Balter M, Downey AB (1997) Exploiting process lifetime distributions for dynamic load balancing. ACM Trans Comput Syst 15(3):253–285Hernandez-Orallo E, Vila-Carbo J (2007) Network performance analysis based on histogram workload models. In: Proceedings of the 15th international symposium on modeling, analysis, and simulation of computer and telecommunication systems (MASCOTS), pp 331–336Hernandez-Orallo E, Vila-Carbo J (2010) Analysis of self-similar workload on real-time systems. In: IEEE real-time and embedded technology and applications symposium (RTAS). IEEE Computer Society, Washington, pp 343–352Hernández-Orallo E, Vila-Carbó J (2010) Network queue and loss analysis using histogram-based traffic models. Comput Commun 33(2):190–201Hughes CJ, Kaul P, Adve SV, Jain R, Park C, Srinivasan J (2001) Variability in the execution of multimedia applications and implications for architecture. SIGARCH Comput Archit News 29(2):254–265Leland W, Ott TJ (1986) Load-balancing heuristics and process behavior. SIGMETRICS Perform Eval Rev 14(1):54–69Leland WE, Taqqu MS, Willinger W, Wilson DV (1994) On the self-similar nature of ethernet traffic (extended version). IEEE/ACM Trans Netw 2(1):1–15Liu CL, Layland JW (1973) Scheduling algorithms for multiprogramming in a hard-real-time environment. J ACM 20(1):46–61Mandelbrot B (1965) Self-similar error clusters in communication systems and the concept of conditional stationarity. IEEE Trans Commun 13(1):71–90Mandelbrot BB (1969) Long run linearity, locally Gaussian processes, h-spectra and infinite variances. Int Econ Rev 10:82–113Norros I (1994) A storage model with self-similar input. Queueing Syst 16(3):387–396Norros I (2000) Queueing behavior under fractional Brownian traffic. In: Park K, Willinger W (eds) Self-similar network traffic and performance evaluation. Willey, New York, Chap 4Park K, Willinger W (2000) Self-similar network traffic: An overview. In: Park K, Willinger W (eds) Self-similar network traffic and performance evaluation. Willey, New York, Chap 1Paxson V, Floyd S (1995) Wide area traffic: the failure of Poisson modeling. IEEE/ACM Trans Netw 3(3):226–244Rolls DA, Michailidis G, Hernández-Campos F (2005) Queueing analysis of network traffic: methodology and visualization tools. Comput Netw 48(3):447–473Rose O (1995) Statistical properties of mpeg video traffic and their impact on traffic modeling in atm systems. In: Conference on local computer networksRoy N, Hamm N, Madhukar M, Schmidt DC, Dowdy L (2009) The impact of variability on soft real-time system scheduling. In: RTCSA ’09: Proceedings of the 2009 15th IEEE international conference on embedded and real-time computing systems and applications. IEEE Computer Society, Washington, pp 527–532Sha L, Abdelzaher T, Årzén KE, Cervin A, Baker T, Burns A, Buttazzo G, Caccamo M, Lehoczky J, Mok AK (2004) Real time scheduling theory: A historical perspective. Real-Time Syst 28(2):101–155Taqqu MS, Willinger W, Sherman R (1997) Proof of a fundamental result in self-similar traffic modeling. SIGCOMM Comput Commun Rev 27(2):5–23Tia T, Deng Z, Shankar M, Storch M, Sun J, Wu L, Liu J (1995) Probabilistic performance guarantee for real-time tasks with varying computation times. In: Proc of the real-time technology and applications symposium, pp 164–173Vila-Carbó J, Hernández-Orallo E (2008) An analysis method for variable execution time tasks based on histograms. Real-Time Syst 38(1):1–37Willinger W, Taqqu M, Erramilli A (1996) A bibliographical guide to self-similar traffic and performance modeling for modern high-speed networks. In: Stochastic networks: Theory and applications, pp 339–366Willinger W, Taqqu MS, Sherman R, Wilson DV (1997) Self-similarity through high-variability: statistical analysis of ethernet lan traffic at the source level. IEEE/ACM Trans Netw 5(1):71–8

Socially and biologically inspired computing for self-organizing communications networks

The design and development of future communications networks call for a careful examination of biological and social systems. New technological developments like self-driving cars, wireless sensor networks, drones swarm, Internet of Things, Big Data, and Blockchain are promoting an integration process that will bring together all those technologies in a large-scale heterogeneous network. Most of the challenges related to these new developments cannot be faced using traditional approaches, and require to explore novel paradigms for building computational mechanisms that allow us to deal with the emergent complexity of these new applications. In this article, we show that it is possible to use biologically and socially inspired computing for designing and implementing self-organizing communication systems. We argue that an abstract analysis of biological and social phenomena can be made to develop computational models that provide a suitable conceptual framework for building new networking technologies: biologically inspired computing for achieving efficient and scalable networking under uncertain environments; socially inspired computing for increasing the capacity of a system for solving problems through collective actions. We aim to enhance the state-of-the-art of these approaches and encourage other researchers to use these models in their future work

Long-range temporal correlations in the EEG bursts of human preterm babies.

The electrical activity in the very early human preterm brain, as recorded by scalp EEG, is mostly discontinuous and has bursts of high-frequency oscillatory activity nested within slow-wave depolarisations of high amplitude. The temporal organisation of the occurrence of these EEG bursts has not been previously investigated. We analysed the distribution of the EEG bursts in 11 very preterm (23-30 weeks gestational age) human babies through two estimates of the Hurst exponent. We found long-range temporal correlations (LRTCs) in the occurrence of these EEG bursts demonstrating that even in the very immature human brain, when the cerebral cortical structure is far from fully developed, there is non-trivial temporal structuring of electrical activity

Long memory estimation for complex-valued time series

Long memory has been observed for time series across a multitude of fields and the accurate estimation of such dependence, e.g. via the Hurst exponent, is crucial for the modelling and prediction of many dynamic systems of interest. Many physical processes (such as wind data), are more naturally expressed as a complex-valued time series to represent magnitude and phase information (wind speed and direction). With data collection ubiquitously unreliable, irregular sampling or missingness is also commonplace and can cause bias in a range of analysis tasks, including Hurst estimation. This article proposes a new Hurst exponent estimation technique for complex-valued persistent data sampled with potential irregularity. Our approach is justified through establishing attractive theoretical properties of a new complex-valued wavelet lifting transform, also introduced in this paper. We demonstrate the accuracy of the proposed estimation method through simulations across a range of sampling scenarios and complex- and real-valued persistent processes. For wind data, our method highlights that inclusion of the intrinsic correlations between the real and imaginary data, inherent in our complex-valued approach, can produce different persistence estimates than when using real-valued analysis. Such analysis could then support alternative modelling or policy decisions compared with conclusions based on real-valued estimation